<html>
<head><meta charset="utf-8"><title>parsing in r-a · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/parsing.20in.20r-a.html">parsing in r-a</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="219612751"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/parsing%20in%20r-a/near/219612751" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> kev <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/parsing.20in.20r-a.html#219612751">(Dec 11 2020 at 14:37)</a>:</h4>
<p>I've just made a summary of the first two phases that matklad drew. Please let me know if you see any mistakes or if it is just wrong. </p>
<p><strong>text (base_db)</strong></p>
<p>Stores the raw text (lexeme and tokens in a database). This is the result of the lexer.</p>
<p><strong>Capabilities</strong>: You can search for tokens and reconstruct the raw source file that you parsed with this.  </p>
<p><strong>Note</strong>: Unlike Some parsers, it stores the whitespace too. The position/span of a node is not explicit in this representation. So with rast, we see <a href="mailto:WHITESPACE@120..121">WHITESPACE@120..121</a> . This can be derived from iterating through all of the nodes on this layer and using their lengths.  </p>
<p><strong>CST (syntax)  </strong></p>
<p>This is the result of the parser. It span information here is relative. So for example <a href="mailto:WHITESPACE@120..121">WHITESPACE@120..121</a> is the span of a whitespace node, but it is relative to all of the other nodes. Therefore, if we insert a whitespace into the file, the CST will be invalidated, but the base_db will not since in the base_db the position is implicit; based on the length of each node.</p>
<p><strong>Capabilities</strong>: There are no new capabilities with this AFAICT. You can search for Expressions and do all of the regular operations you can do on an AST.  </p>
<p><strong>Note</strong>: Since this gets invalidated on every keystroke, it should be very quick to build. It is more or less an inconvenience structure that we only need to build per the file that is open.</p>



<a name="219613184"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/parsing%20in%20r-a/near/219613184" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/parsing.20in.20r-a.html#219613184">(Dec 11 2020 at 14:40)</a>:</h4>
<p>The text query is really just the raw file text, unparsed and unlexed</p>



<a name="219613836"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/parsing%20in%20r-a/near/219613836" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> kev <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/parsing.20in.20r-a.html#219613836">(Dec 11 2020 at 14:45)</a>:</h4>
<p><span class="user-mention silent" data-user-id="211727">Jonas Schievink</span> <a href="#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/parsing.20in.20r-a/near/219613184">said</a>:</p>
<blockquote>
<p>The text query is really just the raw file text, unparsed and unlexed</p>
</blockquote>
<p>Oh I see, thanks for correcting. I thought that this section was about GreenNodes and not salsa</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>